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What quality controls are utilised by PLOS 
ONE when selecting reviewers? Who is 
deemed eligible? 


In 2011, I expressed concern about the PLOS ONE 
business model and its associated review process. 
My worries were focused on the use of what some 
have called light or “lite” peer review, and the “pay- 
to-publish” system used by PLOS ONE (and now 
by many other publishers). My worries were 
subsequently recorded on the PLOS ONE 
Wikipedia page. 


Recent personal experience has increased my 
concern, and left me wondering about the way in 
which reviewers are recruited by PLOS ONE. 


On 12" July, I received an email from a PLOS ONE academic editor 
inviting me to peer review a paper. I won’t say what the paper was entitled, 
or who the authors were, but it was on the topic of open access journals. 


Since I am a blogger/journalist rather than an academic I was surprised to 
receive the invitation, and emailed PLOS ONE with the following question: 
“T have had an invitation to review the above paper. Can you point me to the 
rules on the eligibility of PLOS ONE reviewers?” 


I received the following (I assume boilerplate) reply: 


This did not address my question, so I also emailed the academic editor 
whose name had been at the bottom of the invitation. I am not going to 
name him, but I will say that he is based at Universidad de Las Palmas de 
Gran Canaria, “Was it you who sent me an invitation to review the above 
paper?” I asked. “If so, I am wondering why you invited me. Can you say?” 


I received the following response: 
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Grunt work 


Let me be quite clear at the outset: I had and have no interest whatsoever in 
reviewing this or any other scholarly work, not least because there is 
absolutely no incentive for me to devote my time to reviewing papers. 
Moreover, the one time I did agree to review anything for an academic 
journal (an editorial rather than a paper), my suggestions were all rejected 
on the grounds that “the author says he is too busy to make the changes you 
suggested.” Clearly I had not made very good use of my time! 


But as I say, my first response on receiving the PLOS ONE invitation was to 
wonder whether it is inappropriate for non-academics to review scholarly 
papers. 


With these thoughts in mind I tweeted the invitation under the strapline 
“PLOS ONE invites journalist to review scholarly paper”. Somewhat to my 
surprise, everyone who responded said that they saw no problem with my 
reviewing a scholarly paper on open access (although it could not 
presumably be defined as “peer” review). Their reasoning was that they are 
confident that I have the necessary expertise. And Roger Schonfeld 
commented, “I’d like to see expertise welcomed into the scholarly 


conversation without regard to academic affiliation.” 


On reading these responses I recalled that some OA advocates maintain that 
an important benefit of OA is that it encourages members of the public to 
take a greater interest in science, and to even take part in the process 


themselves — by means of “citizen science”. 


A few days later I read this in the Times Higher: “the open access movement 
in academic publishing, while far from perfect, means that more research 
papers than ever are available to those outside universities. Are we not 
entitled to expect graduates in the workforce and the general public to show 
the curiosity and commitment to engage with research findings relevant to 
them?” 


It added: “Academics should not hide in the ivory tower, but neither should 
non-academics refuse to re-enter it after they graduate. Its doors remain 
wide open. Surely it is not too much to ask for policymakers, journalists, 
non-governmental organisations and industry researchers to make a greater 
effort to access its treasures?” 


That all sounds laudable enough, but I wonder if the authors of the Times 
Higher article (two cultural studies researchers) would want their scholarly 
papers reviewed by members of the public. 


As it happens, I tend these days to turn a somewhat sceptical ear to people 
when they start extolling the virtues of citizen science. As I put it in an 
interview last year, when researchers talk about the benefits of citizen 
science they usually have in mind a scenario in which the public’s role is 
simply to provide free labour for crowdsourcing projects. Citizens are 
despatched to count butterflies, bugs or birds, stare at images of galaxies on 
their computer, or “swab and send”, and then feed the data back to bona fide 
scientists in the lab. 
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“This activity is clearly valuable,” I suggested, “but seems to imply that the 
role of citizens in science is to do the grunt work, while the real science 
continues to be undertaken by professionals.” 


Left wondering 


That said, I was still intrigued as to how and why I had been invited to 
review a paper for PLOS ONE. So I persisted in my attempts to get an 
explanation, putting to PLOS ONE the following three questions: 


Does PLOS ONE allow or not allow people to review a paper where they 
are not a member of a university or other research institution? 


Does PLOS ONE have a published list of the criteria by which reviewers 
can be deemed to be eligible to act as a reviewer? If not, can you tell me 
what they are anyway? 


On what grounds was I invited to review the paper in question, and why was 
it later decided that the invitation was a mistake? 


It was not easy to get an explanation out of PLOS ONE, despite two tweets 
from the publisher saying that it was looking into the matter (here and here). 


gnd 


Eventually (on 22"° July) I received the following email: 


Academic Editors at PLOS ONE have great autonomy when 
selecting reviewers, and the decision as to whether an individual 
expert is eligible to act as a reviewer for a specific manuscript lies 
with the Academic Editor. We have been in touch with [the academic 
editor] and he has noted, as we mentioned, that the reasons why 
specific individuals are approached as reviewers are numerous and 
may vary depending on the submission. In relation to the invitation 
you received [the academic editor] has reiterated that this was 
issued in error, he apologized to you for this error when you first 
raised this to his attention. 


From this I conclude that there is in fact no policy on reviewer eligibility at 
PLOS ONE, although it would have been nice if the publisher had directly 
acknowledged this. 


So I am left wondering how PLOS ONE academic editors make their 
decisions when recruiting reviews. Presumably there will be quite a 
diversity of views on eligibility. Is such a loose arrangement really 
satisfactory? 


Interestingly, a few days later Chealsye Bowley, a librarian at Florida Gulf 
Coast University tweeted about a related problem she had experienced with 
F1000Research. She had been informed that as she does not have a PhD she 
is not eligible to submit a paper to the publishing platform. 


The good news is that F1000Research does have a published authorship 
policy. The better news is that the company responded to Chealsye’s 
concerns by proposing to amend the policy to cater for people like her 
(although some still view the proposed amendment as unsatisfactory). 


Like Chealsye I too do not have a PhD. I did spend a year at Moscow State 
University in 1981 studying the “village prose” writers (e.g. Vasily Belov 
and Valentin Rasputin), but the interminable queues I had to stand in at the 
Lenin Library simply to request access to the books I needed — along with 
the constantly flowing Crommunaa vodka — proved too great a distraction. I 
never finished my thesis. 


Not very encouraging 
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However, the fact that I do not have a PhD would not have been known by 
the PLOS ONE academic editor since I have no CV online. Presumably, 
therefore, he withdrew the invitation when he realised I was not a 
researcher. But should he not have established that before sending me the 
invitation? When I later asked him how the mistake had occurred, and how I 
had been targeted, he did not reply. 


But there are some clues as to how I was selected, and I don’t find them 
very encouraging. Let me explain: 


The invitation I received (below) was sent to me at 12:08 on 12 July 2016. 


PLOS ON 


When I looked at my blog statistics I noted that four minutes earlier 
someone from the University of Las Palmas De Gran Canaria had accessed 
one of the pages on my web site (this one). 


This was the only page the visitor from Gran Canaria accessed that day, so I 
am inferring that the vetting process went something like this: 


The academic editor does a Google search on “open access” and sees my 


name in one of the hits. 
He clicks on the link in that list and is sent to a page on my site. 


He quickly scrolls down that page, copies the email address at the bottom, 
and sends off his invitation. 


This suggests that the academic editor did not think to check whether I was 
a researcher. If so, it is a pretty telling mistake. After all, the banner on the 
top of the page he accessed on my site quite clearly states that I am an 
“independent journalist”. In addition, the email addresses he copied clearly 


describes my status — richard.poynder@journalist.co.uk. 


If that’s right, then the mistake the academic editor made was to spend far 
too little time establishing the eligibility of the person he invited to review 
his open access paper — around four minutes in fact. This is of course 
speculation on my part as the academic editor stopped responding to my 


emails. 
This leaves me with three further questions: 


Do PLOS ONE editors routinely only do four minutes research when 
looking for, and establishing the eligibility of, a reviewer, or was this editor 


in an uncharacteristic hurry? 


Does PLOS ONE have no quality controls in place to enable it to itself 
verify the appropriateness of reviewers being recruited in its name? Does it 
simply leave everything to the academic editor? 
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3. 


If I had not questioned the invitation, but just sent in a review, any old 
review, and that review had met the needs of the academic editor, would 
anyone have known that a “mistake” had been made? After all, PLOS ONE 
does not operate open peer review — its guidelines state: “Reviewers are 
anonymous by default. Reviewers’ identities are not revealed to authors or 
to other reviewers unless reviewers specifically request to be identified by 
signing their names at the end of their comments.” 


Serious issues 


And if the situation at PLOS ONE is as I infer it to be then I think it raises 
some serious issues. As Danny Kingsley has pointed out, problems with 
peer reviewers are now a big, and growing, problem. “In August 2015 
Springer was forced to retract 64 articles from 10 journals, ‘after editorial 
checks spotted fake email addresses, and subsequent internal investigations 


999 


uncovered fabricated peer review reports’”, she wrote recently. “They 


concluded the peer review process had been ‘compromised’.” 


Kingsley added, “In November 2014, BioMed Central uncovered a scam 
where they were forced to retract close to 50 papers because of fake peer 
review issues. This prompted BioMed Central to produce the blog ‘Who 
reviews the reviewers?’ and Nature writing a story on Publishing: the peer 
review scam.” 


Is not PLOS ONE laying itself open to similar scrapes and scandals? (As it 
is, PLOS ONE has already featured on Retraction Watch a number of 
times). 


It is also worth noting that in the Nature article that Kingsley cites 
(published in 2014) the authors point out that the editorial system used by 
PLOS ONE (Editorial Manager) is insecure, since it “actually sends out a 
password, without prompting, whenever it asks a user to sign in, for 
example to review a new manuscript.” Two years later this is still 


happening, as I can now attest. 


All in all, I feel my experience suggests there are two, related, issues here. 
First, there appears to be inadequate checking of reviewers at PLOS ONE, 
and apparently no official policy on reviewer eligibility. 


Second, in describing my invitation as a mistake, PLOS ONE would seem 
to assume that the involvement of citizens in science should, at best, be no 
more than as occasional passive readers of papers, and/or fodder for citizen 
science grunt work. 


As I said, I have absolutely no wish to review papers myself, but I am 
confident that many citizens would be more than willing to contribute. And 
so long as they had the necessary expertise they would probably do a better 
job than many academics (I say that having read quite a few reviews in my 
time). Either way, I would suggest that establishing the expertise of a 
reviewer (whoever they might be) requires more than a quick Google 
search. 


Moreover, if, as many maintain, the boundary between citizens and 
scientists should be more porous, is PLOS ONE sending out the right 
message to the public with incidents like this? I also suspect that had I been 
a researcher I would have received a more detailed and respectful response 
to my enquiries from PLOS ONE. 


The Ivory Tower is dead, long live the Ivory Tower! 


Posted by Richard Poynder at 10:15 


15 comments: 


Anonymous said... 


"...l also suspect that had | been a researcher | would have received a 
more detailed and respectful response to my enquiries from PLOS 
ONE..." 


As a researcher, | find this assumption most amusing! 


July 25, 2016 3:39 pm 


pippasmart said... 


a) | can understand your concern, but as a journal editor myself (Learned 

Publishing), | frequently (almost always?) use non-academics to review 
articles due to the nature of the content. Maybe they do have a PhD, but 
that is irrelevant to me - | choose them because they have knowledge in 
the area that the author has written on and can therefore judge its 
validity. Sometimes | select people specifically because they are not 
experts but | want their opinion as a general reader ("is this 
interesting/sufficiently informative" etc.). 


So, in the case you cite, my assumption is that PLOS had an article 
about a subject on which you have written a great deal and can therefore 
be expected to know a great deal about. | think your suggestion of how 
you were selected was a little disingenuous - | suggest that the editor 
had looked on other sites or spoken to people, discovered your name as 
an "expert", then visited your site to read some of what you had written 
to confirm your knowledge, and then contacted you. 


July 25, 2016 5:24 pm 


Richard Poynder said... 


bs Thank you for your comment Pippa. You too are speculating here of 

course. All | can do is once again invite the academic editor to explain 
the process he went through before inviting me to review the paper, and 
why he then withdrew the invitation when | asked him why he had asked 
me. 


July 25, 2016 5:58 pm 


daniele said... 
a) Dear Richard, thanks a lot for your message. 


Disclaimer: | am an Academic Editor at PLOS One. Academic Editors are 
academics who agree to do some not remunerated work for the journal. 
The reviewers that we choose to assist our decisions are also supposed 
to check whether they have the time, they feel qualified, they have no 
conflicts of interests. Then, they submit a recommendation. Reviewers 
also are not remunerated. 

The ultimate choice is of the Academic Editor. It's certainly a good thing 
to be assisted by reviewers, but it's not mandatory. In the great majority 
of the cases, several reviewers are invited, and usually editors are 
satisfied when they got at least two reviewers who accept to help them. 


The academic editors at PLOS One are thousands, and the journal 
publishes around 30000 papers per year. 


Sorry for this lengthy disclaimer, but | think that on a public platform is 
better to state some things first. It should be clear that here "speaking as 
academic editor of PLOS One" does not mean that | speak on behalf of 
the journal, or that | have any interest beyond contributing to diffusing 
good science. So, | speak as myself, from my experience as Editor as 
PLOS One (and in general). 


Now, thanks again for pointing out a problem. Problems must be spotted 
and solved. 

In this case, your doubt is certainly legitimate, and the Editor should 
have answered your query with something more extensive. We don't 
know what the mistake is, a spelling error, a copy-paste error, or as you 
suggest, an error in considering you suitable or interested when you 
have doubts about it. 


The thing is, as you say, that there is no clear rule for editors to invite 
reviewers. They can rely on their feelings, in the light of receiving the 
most helpful comments. Then it's their job to see if these comments are 


useful or not. 


As an Editor | have received many excellent comments (being then very 
happy of my choice of the reviewers), and some really poor ones (for 
which | said to myself "too bad, your choice was not the good one 
clearly"). 

| also (a couple of times) have invited non academics. The number is too 
low to generalize, but their comments were great, with a fresh 
perspective. 


Regarding your case | think that there was a problem of communication. 
| am interested in whether the editor really thought that you could help, 
on some grounds, and then had an awkward reaction to your question, 
or if he made a trivial mistake and did not mean to invite you. PLOS One 
also uses an automated system to suggest reviewers, based on keyword 
match | think. It's mostly useful, but an editor should always check the 
suitability of the suggested name. 

Anyway none of this can be inferred from the story. 


The thing | am not ok with is the generalization. | was pointed to your 
article by a twitter user who titled "The way in which reviewers are 
recruited at PLOS One". Not your responsibility of course. 

But | think that you shift the attention from a single, actual problem, when 
you put in the same post worries about pay-per-publish. This system is 
good for open access, but has some problems, as those pointed out by 
Brembs here http://bjoern.brembs.net/2016/04/how-gold-open-access- 
may-make-things-worse/ 


but | really don't think it's connected to the quality of the papers. 


Also the presence on Retraction Watch of PLOS One papers is 
anecdotal, given the huge number of published papers. The number of 
retractions is indeed the only thing that clearly correlates with Impact 
Factor, being led by glamour (and overly respected) journals. 


http://journal.frontiersin.org/article/10.3389/fnhum.2013.00291/full 


Most importantly, Open Access itself is not correlated with bad or sloppy 
reviews (nor is PLOS One to be associated with Open Access as a 
whole). 


Anyway, thanks again for pointing to the communication problem that 
you had with one of the 6000 editors. 
Sorry for the lengthy comment, | hope it's useful. 


All the best, and keep up the good work. 


Daniele Marinazzo 


July 25, 2016 9:39 pm 


Stevan Harnad said... 


Generic ”Megajournals”? 
My 2p worth: 


(1) You, Richard, are an appropriate reviewer for some papers on open 
access or on publishing. (PhDs and academic status have nothing to do 
with it.) | would have chosen you for some papers, if | were editor. 


(2) PloS ONE is not a journal: Nothing that has thousands of editors and 
publishes 30,000 journals yearly is a journal. 


(3) You imagine that peer review is a more solemn and systematic a 
process than it actually is: There is huge variation in its rigour from 
journal to journal and discipline to discipline. 


(4) But at least with a journal, with a track-record, what you see is what 
you are likely to get. 


(5) With PloS ONE you don’t just get “peer review lite” (whatever that 
means): You get a pine-nut in a poke. [I abjure the usual sociopathic 
cliché.] Impossibl 


July 26, 2016 6:40 pm 


Stevan Harnad said... 


Correction above: 


Nothing that has thousands of editors and publishes 30,000 papers 
yearly is a journal. 


July 26, 2016 7:48 pm 


Stevan Harnad said... 


Correction #2 above: 


(5) Impossible to know what you're getting. 


July 26, 2016 7:50 pm 


Mike Taylor said... 


Seems to me like a classic case of "Move along, nothing to see". Here is 
what | see as the likely scenario: 

1. Editor invited to handle a paper about open access. 

2. Invites a recognised expert on the subject to review. 

3. Gets a slightly snotty message asking about it. 

4. Thinks "I can do without this, I'll find someone else" and withdraws the 
invitation. 

| imagine | would have done the same in his position. 


For the record: | firmly believe that reviewers can and should be drawn 
from the entire population of those with relevant expertise, completely 
irrespective of their formal qualifications or affiliations. And F1000's 
"must have a Ph.D to be an author" policy is a horrifyingly elitist blunt 
instrument, which leaves me feeling absolutely no inclination ever to 
send my work to them. 


July 26, 2016 9:21 pm 


Richard Poynder said... 


Thanks for your comment Mike. Your scenario is more speculation of 
course, which reminds us that one thing the open access movement has 
generally failed to do is to rid us of the secrecy, and the unaccountable 
gatekeepers, that have for so long plagued scholarly communication. At 
least F1000 is trying to increase transparency of process. 


July 27, 2016 6:40 am 


Klaas van Dijk said... 


My experiences until now with communicating with members of the staff 
of publisher PLOS show similarities with the experiences of Richard 
Poynder. | have received several so-called boilerplate responses (or no 
response at all) and | have received loads and loads of auto-replies. 


All of these auto-replies have a different case ID and all of them state 
something like "We will respond to your email as soon as we are able". 
This simply does not happen. 


See https://scholarlyoa.com/2016/01/14/another-controversial-paper- 
from-fronters/#comment-405567 for some details of my experiences with 
Iratxe Puebla (currently the Managing Editor at PLOS ONE). 


PLOS has an extensive policy for authors, reviewers, editors and 
readers for all journals to declare competing interests, see 
http://journals.plos.org/plosone/s/competing-interests 


Members of the staff of the journal PLOS ONE do not need to disclose 
such details. Iratxe Puebla is for example acting as a consultant for 
COPE, but this information is not listed at 
http://journals.plos.org/plosone/s/staff-editors 


July 27, 2016 7:20 am 


Karen Shashok said... 


People with no academic or institutional affiliation can have knowledge, 
experience and expertise their age- and discipline-matched counterparts 
in the Ivory Tower may lack. It’s healthy for reviewers and editors to seek 
expertise outside of academia if they think someone is likely to have 
useful feedback to offer. 


| think this was the situation since Open and Shut is awesome and very 
well known among those who follow developments in academic 
publishing and open access. You were given a golden opportunity to help 


some researchers, somewhere, improve their article. You 
underestimated your skills, or chose to focus on a non-issue. Nice 
mountain! Too bad it’s built out of a molehill. 


Your PLoS ONE academic editor did not respond to you in a way you 
expected, and your reaction is understandable — but is not a reaction that 
other non-academic invited reviewers would share, | think. When I’ve 
been invited to review for a journal I’ve always felt honored and delighted 
to help. (My highest academic degree is a BA and I’ve been self- 
employed all my professional life. That hasn’t been an issue so far for 
editors who’ve contacted me.) If | see from the invitation and abstract 
that | don’t have the necessary expertise to contribute useful feedback, | 
decline the invitation, and that’s the end of it. Takes maybe 5 minutes. 


The PLoS ONE system of editorial quality control is decentralized, and 
there are no detailed SOPs for their thousands of experts who provide 
unremunerated editorial support. | think PLoS is aware that the lack of 
transparent information about their processes worries people, and are 
taking steps to provide more information. Meanwhile there will be 
considerable variation in how individual academic editors and reviewers 
handle their appointed tasks. 


But even at journals with fine-grained review procedures (some but not 
so many) that are fully reported on their website (not many at all), human 
errors and oversights occur — ask anyone who publishes in journals or 
peer reviews for them. This is nothing to do with the PLoS model or with 
open access. 


(BTW, what need was there to identify the institution you surmised the 
academic editor is from? Are you suggesting that it has anything to do 
with this person’s competence or behavior? | found this off-putting, but 
maybe that’s just another molehill.) 


August 02, 2016 11:39 am 


Richard Poynder said... 


Dear Karen, 
Thanks for taking the time to post a comment. Just to clarify: 


1. | have no desire or wish to review any scholarly paper, be it published 
by PLOS ONE or any other scholarly publisher. As such, there was no 
golden opportunity here that | passed up. 


2. You say that when you are invited to review papers you take a view on 
whether or not you are qualified to review them. In my case this was not 
possible as the abstract was not sufficient for me to make that decision 
(had | been interested in reviewing the paper), and when | logged into 
Editorial Manager to form a better view of what the paper was about the 
system did not let me proceed unless | filled in a box saying what 
institution |am a member of. This was problematic as | am not a member 
of an institution. 


3. | don’t believe that the points | made are a non-issue. As you yourself 
acknowledge, there are at the very least problems with transparency in 
PLOS ONE'’s processes. You say that PLOS ONE is aware of this and is 
taking steps to provide more information. Can you clarify this? Are you 
speaking for or about PLOS ONE? 


4. | have no problem in principle with decentralised systems, but 
decentralisation without transparency appears to be problematic. 


5. You say that | surmised who the academic editor is. | did not surmise 
this, his name was on the invitation | received, and PLOS ONE 
confirmed the name of the institution he works for. 


6. There was no intention whatsoever to suggest that the institution 
concerned has any bearing on the academic editor’s competence or 
behaviour, nor do | believe it does. | identified the university simply in 
order to support my interpretation of how and why | was invited, which if 
correct is worrying. 


7. You say that even journals with fine-grained review procedures 
experience human errors. My point in the post was not about human 
errors but the nature of the PLOS ONE system. | am concerned about its 
non-transparent processes; the huge number of papers it publishes (and 
the consequent high degree of automated processes this requires); the 


way the business model encourages a high number of papers to be 
submitted to it; and an unwillingness to engage with people who 
experience problems and/ or have concerns with PLOS ONE. 


8. If it is true that | have made a mountain out of a molehill, why have the 
academic editor and PLOS ONE consistently declined to explain why | 
was invited and then told me that my invitation was a mistake? Why 
would they not want to explain it, if only privately? 


August 03, 2016 9:40 am 


Richard Poynder said... 


See also this post: https://scholarlyoa.com/2016/08/23/ongoing- 
questions-about-plos-ones-peer-review/ 


August 24, 2016 1:17 pm 


Unknown said... 


Your post raises some valid points, but as someone who has been 
involved in academic publishing in many roles (author, reviewer, editorial 
board member, "academic editor", editor-in-chief) for both OA and 
subscription journals, there are a few remarks I'd like to make. 


1/1 do not know of ANY journal that has a stated policy about who can 
be a reviewer or not I'm not saying they don’ exist, but I'm sure it is 
something quite rare). 


2/ Some of the best reviews | received as an editor came from graduate 
students without a PhD. Some of the worst reviews | got came from full 
professors at reputable universities, who just should have said "no" 
because they clearly lacked the necessary time to do a thorough job and 
thus missed the most obvious fatal flaws. 


3/ | know a lot of idiots with a PhD. | know a lot of qualified people that 
don't have one. Although in most cases an editor will invite a reviewer 
who does have a PhD, this is certainly not a rule written in stone and, as 
far as | am concerned, totally irrelevant. What counts is whether the 
reviewer has the necessary expertise. 


4/ Every editor makes mistakes. If they do this often, they're simply bad 
editors and should be removed. But even the best editors are only 
human. 


5/ You underestimate yourself. You're a well-known specialist on open 
access. | would not accept to edit an article on OA myself as | do not 
think I'm qualified for that, but if ever | did, | would not see any problem 
with inviting a recognized specialist, even if he has no PhD or formal 
academic affiliation. | am sorry that you let one unfortunate experience 
make you unwilling to even consider reviewing a paper. 


6/ PLOS ONE is by now the largest journal around, publishing a 
significant proportion of all papers indexed in, for example, MEDLINE. 
Given the sheer number of articles published, | think the number of 
retractions is actually below the mean of most journals. They have over 
5000 academic editors and, again, given that huge number some will be 
sub-par. As far as | know, really weak editors get weeded out, but that 
means of course that by the time this happens, they have made some 
mistakes. However, | don't really see a way to do this better... 


August 29, 2016 1:47 pm 


Unknown said... 


| agree that the editor's response is difficult to understand. | don't see 
anything wrong with asking a non-academic to review a paper in 
principle -- though | haven't found reason to do that yet. If the person 
doesn't respond in a given time, then the answer is "no". If the person 
complains, say "I'm sorry for troubling you" (if nothing else) and move on. 


| would like to add a little to Wim Crusio's point 3 (above). 


| think following comment somewhat misses the point of scholarly 
publishing 


“This activity is clearly valuable,” | suggested, “but seems to imply that 
the role of citizens in science is to do the grunt work, while the real 
science continues to be undertaken by professionals.”" 


This probably is true in practice. Nevertheless, good science and good 
scholarship are not (or at least should not be) limited to academics. We 
city slickers may say "dumb farmer"; however, a successful farmer has to 
be pretty smart, in fact. 


We (as academics) are better trained on how to communicate in such a 
medium (compared to citizen science), but quality scholarship can (and 
should) be recognized wherever it comes from. To carry the farmer 
example further, the chance to stumble an interesting topic to write on, 
the challenge of obtaining the research materials (specific types of crops 
perhaps, and then to navigate through pay walls and limited library 
support), the difficulty of understanding such specialized (and often 
Opaque) papers, and the challenge of writing an original paper of one's 
own in a style compatible with an academic publication are all things that 
would make it difficult for a farmer in Indiana on his own dime to 
produce. 


We should not be prejudice toward a work simply because it was written 
by a farmer (to carry this example). 


Moreover, when | have reviewed very poorly written papers by some 
academics, | often comment to those authors to consider the janitor in 
some obscure part of the world picking up this paper out of a waste can. 
What would s/he think? Consider the student who is wandering through 
the stacks one day and just happens to stumble on this paper when 
opening a random booklet. (| guess now, it would be surfing the web, but 
| think the point is clear.) We don't know who is going to read our work. 
What would that student think? 


Papers should not be written with the mind that no one other than some 
other very closely allied specialist is going to read that work. Indeed, this 
is the most exclusionary of all writing. Newspapers have to make some 
attempt to connect their readers. Likewise, journal publications should 
make at least a feeble attempt to provide a context and connect with the 
reader. As Strunk & White wrote, "we should consider that the reader 
needs help" (and probably lots of it). As far as we have the ability, our 
writing should be for the readers, and we should never see these 
publications as little more than trophies to be stacked up for use in 
gaining further academic promotions. 


Consider what Benjamin Franklin wrote: 

"If my hypothesis is not the truth, it is at least naked; 
for | have not with some of our learned moderns 
disguis'd my nonsense in Greek, 


cloth'd it in algebra or adorned it with fluxions." 


Franklin, by the way, was not particularly an traditional "academic". 
August 29, 2016 4:12 pm 
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